Time critical tasks scheduling

ABSTRACT

A method and system for scheduling a time critical task. The system may include a processing unit, a hardware assist scheduler, and a memory coupled to both the processing unit and the hardware assist scheduler. The method may include receiving timing information for executing the time critical task, the time critical task executing program instructions via a thread on a core of a processing unit and scheduling the time critical task based on the received timing information. The method may further include programming a lateness timer, waiting for a wakeup time to obtain and notifying the processing unit of the scheduling. Additionally, the method may include executing, on the core of the processing unit, the time critical task in accordance with the scheduling, monitoring the lateness timer, and asserting a thread execution interrupt in response to the lateness timer expiring, thereby suspending execution of the time critical task.

PRIORITY DATA

This application is a continuation of U.S. patent application Ser. No.13/859,030, titled “A Hardware Assisted Method and System for SchedulingTime Critical Tasks”, filed Apr. 9, 2013, whose inventors are SundeepChandhoke and Herbert K. Salmon, IV, which is hereby incorporated byreference in its entirety as though fully and completely set forthherein.

FIELD OF THE INVENTION

The present invention relates to a hardware assisted method and systemfor scheduling time critical tasks.

DESCRIPTION OF THE RELATED ART

Real time systems use time critical tasks to execute programinstructions that are required to meet deadlines, e.g., to not run late.Not meeting a deadline is considered catastrophic in hard real timesystems. Examples of hard real time systems include engine controlsystems, industrial process controllers, and various embedded systemswhich interact at a low level with physical hardware.

There are many techniques for scheduling time critical tasks. A realtime scheduler is a critical part of a real time system. It is theresponsibility of the real time scheduler to execute tasks in a timelymanner. An essential function of the real time scheduler is monitoringscheduled time critical tasks to ensure the time critical tasks aremeeting their deadline. Additionally, the real time scheduler mustensure fairness in use of resources of the real time system. Forexample, in addition to time critical tasks, all real time systems usebest effort tasks, e.g., tasks that have no timeliness requirement, toexecute non time critical programs such as logging status, responding touser interface commands, monitoring health of the system, and so forth.Hence, it is the responsibility of the real time scheduler to ensurethat, although a processing unit's resources are shared between besteffort tasks and time critical tasks, the timeliness of time criticaltasks is not compromised by the execution of the best effort tasks.Therefore, the real time scheduler is required to pre-empt, orinterrupt, the best effort tasks when it is time to execute the timecritical tasks again.

However, this functionality of the real time scheduler consumes preciousprocessing unit cycles, especially in instances where it is required toeither run multiple time critical tasks, execute the time critical taskat high rates, e.g., high frequency cycles, or both. Hardware assistedmechanisms are available which allow the offloading of schedulercalculations to a separate device such as a peripheral device connectedto the processing unit via the processor interconnect, e.g., an fieldprogrammable gate array (FPGA) or another processing unit. Additionally,non-interrupt based scheduling to improve processor unit usage byelimination of the overhead associated with interrupts is available.

However, there is a need for a real time scheduler that monitorsdeadlines for time critical tasks while ensuring fairness in processingunit resource sharing between the time critical tasks and best efforttasks without compromising timeliness in waking up the time criticaltasks.

SUMMARY OF THE INVENTION

Various embodiments of a system and method for scheduling time criticaltasks are presented below.

The method for monitoring a time critical task may include receivingtiming information for executing the time critical task, where the timecritical task may execute program instructions via a thread on a core ofa processing unit. The method may further include scheduling the timecritical task, where the scheduling may include determining, based onthe received timing information, a wakeup time, a thread cycle time, anoverall thread deadline, and a time critical task deadline. In someembodiments, the overall thread deadline may be the difference betweenthe thread cycle time and a system dependent suspend latency. In certainembodiments, the time critical task deadline may be a function of theoverall thread deadline and a specified thread bandwidth, where thespecified thread bandwidth may indicate a fraction of the thread cycletime reserved for the time critical task.

The method may further include programming a timer, e.g., a latenesstimer, based on the wakeup time and the time critical task deadline, andwaiting for the wakeup time to obtain, e.g., waiting for the wakeup timeto arrive. Further, the method may include notifying the processing unitof the scheduling, where the notifying may indicate to the processingunit that the thread is in an idle state. The idle state may indicatethat the thread is awake and ready to resume execution and execute thetime critical task in accordance with the scheduling.

Additionally, the method may include executing, on the core of theprocessing unit, the time critical task in accordance with thescheduling. Further, the method may include monitoring the lateness timeand asserting a thread interrupt in response to the lateness timerexpiring, where asserting the thread interrupt interrupts the executing,thereby suspending execution of the time critical task. In certainembodiments, suspending execution of the time critical task may includesuspending execution of the thread and returning the thread to the idlestate, thereby ensuring a timely wakeup of the thread for execution ofanother cycle of the time critical task.

In some embodiments, notifying the processing unit of the scheduling mayinclude a memory write to a memory of the processing unit. In suchembodiments, the method may further include monitoring the memory, bythe processing unit, for the memory write to indicate the notifying ofthe scheduling.

In other embodiments, notifying the processing unit of the schedulingmay include a timed thread execution. The timed thread execution mayinclude synchronizing, via a time monitor circuit, a clock of theprocessing unit with a global clock, monitoring, by the processing unitvia the time monitor circuit, the clock of the processing unit withrespect to the wakeup time, and determining that the wakeup timeobtains. In such embodiments, the method may further include resumingthe execution of the thread in response to determining that the wakeuptime obtains.

In some embodiments the thread may include a multiplexed thread, wherethe multiplexed thread is executed on the core of the processing unit.In certain embodiments, the multiplexed thread may include a pluralityof time critical tasks including the time critical task, where each ofthe plurality of time critical tasks has respective timing information,and where each of the plurality of time critical tasks is executed inaccordance with the respective timing information. In other embodiments,the multiplexed thread may include one or more time critical tasks and abest effort task, e.g., a non-time critical task. In such embodiments,the method may further include waking up the best effort task inresponse to the lateness timer expiring, executing the best effort taskuntil the overall thread deadline obtains, resetting the lateness timer,suspending the thread and returning the thread to the idle state,thereby ensuring a timely wakeup of the thread for execution of anothercycle of the time critical task. In certain embodiments, the method mayalso include receiving timing information for executing the best efforttask and scheduling the best effort task. The scheduling of the besteffort task may include determining, based on the received timinginformation, a start time for the best effort task, where the start timemay be later than the time critical task deadline.

Additionally, a system for scheduling a time critical task may beprovided that may implement embodiments of the methods described herein.The system may include a processing unit, a device configured as ahardware assist scheduler, and a memory coupled to the processing unitand the hardware assist scheduler.

The hardware assist scheduler may be configured to receive timinginformation for executing the time critical task, where the timecritical task executes program instructions via a thread on a core ofthe processing unit. Further, the hardware assist scheduler may beconfigured to schedule the time critical task. To schedule the timecritical task, the device may determine, based on the received timinginformation, a wakeup time, a thread cycle time, an overall threaddeadline, and a time critical task deadline. In some embodiments, theoverall thread deadline may be the difference between the thread cycletime and a system dependent suspend latency. Additionally, in certainembodiments, the time critical task deadline may be a function of theoverall thread deadline and a specified thread bandwidth, where thespecified thread bandwidth may indicate a fraction of the thread cycletime reserved for the time critical task.

Further the hardware assist scheduler may be configured to program alateness timer based on the wakeup time and the time critical taskdeadline and wait for the wakeup time to obtain, e.g., wait for thewakeup time to arrive.

Additionally, the hardware assist scheduler may be configured to notifythe processing unit of the schedule. To notify the processing unit ofthe schedule, the hardware assist scheduler may be configured toindicate to the processing unit that the thread is in an idle state,where the idle state may indicate that the thread is awake and ready toresume execution and execute the time critical task in accordance withthe schedule. The hardware assist scheduler may also be configured tomonitor the lateness timer and assert a thread interrupt in response tothe lateness timer expiring. To assert the thread interrupt, thehardware assist scheduler may be configured to interrupt execution ofthe thread on the core of the processing unit, thereby suspendingexecution of the time critical task. In some embodiments, to suspendexecution of the time critical task, the hardware assist scheduler maybe further configured to suspend execution of the thread and return thethread to the idle state, thereby ensuring a timely wakeup of the threadfor execution of another cycle of the time critical task.

The processing unit may be configured to resume execution of the threadupon notification that the thread is in the idle state and execute thetime critical task on the thread.

In some embodiments, to notify the processing unit of the schedule, thehardware assist scheduler may be further configured to perform a memorywrite to the memory. In such embodiments, the processing unit may befurther configured to monitor the memory for the memory write.

In other embodiments, to notify the processing unit of the schedule, thehardware assist scheduler may further configured to provide a globalclock signal. In such embodiments, the processing unit may be furtherconfigured to perform a timed thread execution. The processing unit toperform the timed thread execution, may be configured to synchronize,via a time monitor circuit, a clock of the processing unit with theglobal clock, monitor, via the time monitor circuit, the clock of theprocessing unit with respect to the wakeup time, determine that thewakeup time obtains, and resume the execution of the thread in responseto the wakeup time obtaining.

In certain embodiments, the thread may include a multiplexed thread,where the multiplexed thread is executed on the core of the processingunit. In some embodiments the multiplexed thread may include a pluralityof time critical tasks including the time critical task, where each ofthe plurality of time critical tasks has respective timing information,and where each of the plurality of time critical tasks is executed inaccordance with the respective timing information.

In other embodiments, the multiplexed thread may include one or moretime critical tasks and a best effort task. In such embodiments, thehardware assist scheduler may be further configured to wake up the besteffort task in response to the lateness timer expiring, notify theprocessing unit that the best effort task is awake and ready to beexecuted, reset the lateness timer, suspend the thread, and return thethread to the idle state, thereby ensuring a timely wakeup of the threadfor execution of another cycle of the time critical task. In anexemplary embodiment, the hardware assist scheduler may be furtherconfigured to receive timing information for executing the best efforttask and schedule the best effort task. To schedule the best efforttask, the hardware assist scheduler may be configured to determine,based on the received timing information, a start time for the besteffort task, where the start time may be later than the time criticaltask deadline.

In some embodiments, the processing unit may include a plurality ofcores. In such embodiments, the hardware assist scheduler may include acore of the plurality of cores of the processing unit. Additionally, thetime critical tasks may execute on at least one other core of theplurality of cores of the processing unit. In other embodiments, theprocessing unit may include a first processing unit, and the hardwareassist scheduler may include a second processing unit.

In certain embodiments, the system may further include an interconnect,where the interconnect communicatively couples the hardware assistscheduler to the memory and to the processing unit. In some embodiments,the interconnect may include a high speed interconnect. In otherembodiments, the interconnect may include a photonics interconnect.

BRIEF DESCRIPTION OF THE DRAWINGS

A better understanding of the present invention can be obtained when thefollowing detailed description of the preferred embodiment is consideredin conjunction with the following drawings, in which:

FIG. 1A illustrates a computer system configured to execute programinstructions according to an embodiment of the present invention;

FIG. 1B illustrates a network system comprising two or more computersystems that may implement an embodiment of the present invention;

FIG. 2A illustrates an instrumentation control system that may implementan embodiment of the present invention;

FIG. 2B illustrates an industrial automation system that may implementan embodiment of the present invention;

FIG. 3 is an exemplary block diagram of a computer system, according toan embodiment of the present invention;

FIG. 4 is an exemplary block diagram of a computer system, according toan embodiment of the present invention;

FIG. 5 is an exemplary block diagram of a computer system, according toan embodiment of the present invention;

FIG. 6 is an exemplary block diagram of a processing unit, according toan embodiment of the present invention;

FIG. 7 is an exemplary block diagram of a computer system, according toan embodiment of the present invention; and

FIG. 8 is an exemplary flow chart of a method for implementing anembodiment of the present invention.

While the invention is susceptible to various modifications andalternative forms, specific embodiments thereof are shown by way ofexample in the drawings and are herein described in detail. It should beunderstood, however, that the drawings and detailed description theretoare not intended to limit the invention to the particular formdisclosed, but on the contrary, the intention is to cover allmodifications, equivalents and alternatives falling within the spiritand scope of the present invention as defined by the appended claims.

DETAILED DESCRIPTION OF THE INVENTION Incorporation by Reference

The following references are hereby incorporated by reference in theirentirety as though fully and completely set forth herein:

U.S. patent application Ser. No. 13/594,088, entitled “Hardware AssistedReal-time Scheduler Using Memory Monitoring” filed on Aug. 24, 2012.

U.S. Provisional Patent Application No. 61/714,006, entitled “TimeMonitoring in a Processing Element and Use” filed on Oct. 15, 2012.

TERMS

The following is a glossary of terms used in the present application:

Memory Medium—Any of various types of memory devices or storage devices.The term “memory medium” is intended to include an installation medium,e.g., a CD-ROM, floppy disks 104, or tape device; a computer systemmemory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM,Rambus RAM, etc.; a non-volatile memory such as a Flash, magnetic media,e.g., a hard drive, or optical storage; registers, or other similartypes of memory elements, etc. The memory medium may comprise othertypes of memory as well or combinations thereof. In addition, the memorymedium may be located in a first computer in which the programs areexecuted, or may be located in a second different computer whichconnects to the first computer over a network, such as the Internet. Inthe latter instance, the second computer may provide programinstructions to the first computer for execution. The term “memorymedium” may include two or more memory mediums which may reside indifferent locations, e.g., in different computers that are connectedover a network.

Carrier Medium—a memory medium as described above, as well as a physicaltransmission medium, such as a bus, network, and/or other physicaltransmission medium that conveys signals such as electrical,electromagnetic, or digital signals.

Programmable Hardware Element—includes various hardware devicescomprising multiple programmable function blocks connected via aprogrammable interconnect. Examples include FPGAs (Field ProgrammableGate Arrays), PLDs (Programmable Logic Devices), FPOAs (FieldProgrammable Object Arrays), and CPLDs (Complex PLDs). The programmablefunction blocks may range from fine grained (combinatorial logic or lookup tables) to coarse grained (arithmetic logic units or processorcores). A programmable hardware element may also be referred to as“reconfigurable logic”.

Program—the term “program” is intended to have the full breadth of itsordinary meaning. The term “program” includes 1) a software programwhich may be stored in a memory and is executable by a processor or 2) ahardware configuration program useable for configuring a programmablehardware element.

Software Program—the term “software program” is intended to have thefull breadth of its ordinary meaning, and includes any type of programinstructions, code, script and/or data, or combinations thereof, thatmay be stored in a memory medium and executed by a processor. Exemplarysoftware programs include programs written in text-based programminglanguages, such as C, C++, PASCAL, FORTRAN, COBOL, JAVA, assemblylanguage, etc.; graphical programs (programs written in graphicalprogramming languages); assembly language programs; programs that havebeen compiled to machine language; scripts; and other types ofexecutable software. A software program may comprise two or moresoftware programs that interoperate in some manner. Note that variousembodiments described herein may be implemented by a computer orsoftware program. A software program may be stored as programinstructions on a memory medium.

Hardware Configuration Program—a program, e.g., a netlist or bit file,that can be used to program or configure a programmable hardwareelement.

Computer System—any of various types of computing or processing systems,including a personal computer system (PC), mainframe computer system,workstation, network appliance, Internet appliance, personal digitalassistant (PDA), television system, grid computing system, or otherdevice or combinations of devices. In general, the term “computersystem” can be broadly defined to encompass any device (or combinationof devices) having at least one processor that executes instructionsfrom a memory medium.

Measurement Device—includes instruments, data acquisition devices, smartsensors, and any of various types of devices that are configured toacquire and/or store data. A measurement device may also optionally befurther configured to analyze or process the acquired or stored data.Examples of a measurement device include an instrument, such as atraditional stand-alone “box” instrument, a computer-based instrument(instrument on a card) or external instrument, a data acquisition card,a device external to a computer that operates similarly to a dataacquisition card, a smart sensor, one or more DAQ or measurement cardsor modules in a chassis, an image acquisition device, such as an imageacquisition (or machine vision) card (also called a video capture board)or smart camera, a motion control device, a robot having machine vision,and other similar types of devices. Exemplary “stand-alone” instrumentsinclude oscilloscopes, multimeters, signal analyzers, arbitrary waveformgenerators, spectroscopes, and similar measurement, test, or automationinstruments.

A measurement device may be further configured to perform controlfunctions, e.g., in response to analysis of the acquired or stored data.For example, the measurement device may send a control signal to anexternal system, such as a motion control system or to a sensor, inresponse to particular data. A measurement device may also be configuredto perform automation functions, i.e., may receive and analyze data, andissue automation control signals in response.

Functional Unit (or Processing Element)—refers to various elements orcombinations of elements. Processing elements include, for example,circuits such as an ASIC (Application Specific Integrated Circuit),portions or circuits of individual processor cores, entire processorcores, individual processors, programmable hardware devices such as afield programmable gate array (FPGA), and/or larger portions of systemsthat include multiple processors, as well as any combinations thereof.

Automatically—refers to an action or operation performed by a computersystem (e.g., software executed by the computer system) or device (e.g.,circuitry, programmable hardware elements, ASICs, etc.), without userinput directly specifying or performing the action or operation. Thusthe term “automatically” is in contrast to an operation being manuallyperformed or specified by the user, where the user provides input todirectly perform the operation. An automatic procedure may be initiatedby input provided by the user, but the subsequent actions that areperformed “automatically” are not specified by the user, i.e., are notperformed “manually”, where the user specifies each action to perform.For example, a user filling out an electronic form by selecting eachfield and providing input specifying information (e.g., by typinginformation, selecting check boxes, radio selections, etc.) is fillingout the form manually, even though the computer system must update theform in response to the user actions. The form may be automaticallyfilled out by the computer system where the computer system (e.g.,software executing on the computer system) analyzes the fields of theform and fills in the form without any user input specifying the answersto the fields. As indicated above, the user may invoke the automaticfilling of the form, but is not involved in the actual filling of theform (e.g., the user is not manually specifying answers to fields butrather they are being automatically completed). The presentspecification provides various examples of operations beingautomatically performed in response to actions the user has taken.

Time Critical Task—refers to a task in a real time computer system usedto execute program instructions that need to meet deadlines, e.g., notrun late.

Best Effort Task—refers to a task in a real time computer system used toexecute program instructions that does not have a timelinessrequirement, e.g., do not need to meet a deadline. Examples include, butare not limited to, non-time critical programs such as logging status,responding to user interface commands, monitoring health of the system,and so forth.

Thread—refers to the smallest sequence of programming instructions thatan operating system scheduler can manage independently.

FIG. 1A—Computer System

FIG. 1A illustrates a computer system 82 configured to execute programinstructions according to an embodiment of the present invention. Oneembodiment of a method for implementing an embodiment of the presentinvention is described below in reference to FIG. 10.

As shown in FIG. 1A, the computer system 82 may include a display deviceconfigured to display a graphical program as the graphical program iscreated and/or executed. The display device may also be configured todisplay a graphical user interface or front panel of the graphicalprogram during execution of the graphical program. The graphical userinterface may comprise any type of graphical user interface, e.g.,depending on the computing platform.

The computer system 82 may be configured as a system for scheduling atime critical task and may include at least one processing element, adevice configured as a hardware assist scheduler and a memory coupled tothe processing element and the hardware assist scheduler according toone embodiment of the present invention. Examples of a processingelement include, but are not limited to, circuits such as an ASIC(Application Specific Integrated Circuit), portions or circuits ofindividual processor cores, entire processor cores, individualprocessors, such as processing units, programmable hardware devices suchas a field programmable gate array (FPGA), and/or larger portions ofsystems that include multiple processors, as well as any combinationsthereof. Additionally, a hardware assist scheduler may be implemented onany type of processing element, either as an integral part of thecomputing system or as a peripheral device communicatively coupled tothe computer system via an interconnect.

FIG. 1B—Computer Network

FIG. 1B illustrates a system including a first computer system 82 thatis coupled to a second computer system 90. The computer system 82 may becoupled via a network 84 (or a computer bus) to the second computersystem 90. The computer systems 82 and 90 may each be any of varioustypes, as desired. The network 84 can also be any of various types,including a LAN (local area network), WAN (wide area network), theInternet, or an Intranet, among others. According to various embodimentsof the present invention, a system for scheduling a time critical taskmay include a processing unit included in computer system 82 and adevice configured as a hardware assist scheduler included in computersystem 90 where the processing unit and hardware assist scheduler arecoupled to a memory included in either computer system 82 or computersystem 90 via an interconnect.

Exemplary Systems

Embodiments of the present invention may be involved with performingtest and/or measurement functions; controlling and/or modelinginstrumentation or industrial automation hardware; modeling andsimulation functions, e.g., modeling or simulating a device or productbeing developed or tested, etc. Exemplary test applications where thesystem for scheduling a time critical task may be used includehardware-in-the-loop testing and rapid control prototyping, amongothers.

However, it is noted that embodiments of the present invention can beused for a plethora of applications and is not limited to the aboveapplications. In other words, applications discussed in the presentdescription are exemplary only, and embodiments of the present inventionmay be used in any of various types of systems. Thus, embodiments of thesystem and method of the present invention is configured to be used inany of various types of applications, including the control of othertypes of devices such as multimedia devices, video devices, audiodevices, telephony devices, Internet devices, etc., as well as generalpurpose software applications such as word processing, spreadsheets,network control, network monitoring, financial applications, games, etc.

FIG. 2A illustrates an exemplary instrumentation control system 100which may implement embodiments of the invention. The system 100comprises a host computer 82 which couples to one or more instruments.The host computer 82 may comprise a CPU, which may be configured withvarious embodiments of the present invention, a display screen, memory,and one or more input devices such as a mouse or keyboard as shown. Thecomputer 82 may operate with the one or more instruments to analyze,measure or control a unit under test (UUT) or process 150, e.g., viaexecution of software 104.

The one or more instruments may include a GPIB instrument 112 andassociated GPIB interface card 122, a data acquisition board 114inserted into or otherwise coupled with chassis 124 with associatedsignal conditioning circuitry 126, a VXI instrument 116, a PXIinstrument 118, a video device or camera 132 and associated imageacquisition (or machine vision) card 134, a motion control device 136and associated motion control interface card 138, and/or one or morecomputer based instrument cards 142, among other types of devices. Thecomputer system may couple to and operate with one or more of theseinstruments. The instruments may be coupled to the unit under test (UUT)or process 150, or may be coupled to receive field signals, typicallygenerated by transducers. The system 100 may be used in a dataacquisition and control application, in a test and measurementapplication, an image processing or machine vision application, aprocess control application, a man-machine interface application, asimulation application, or a hardware-in-the-loop validationapplication, among others.

FIG. 2B illustrates an exemplary industrial automation system 200 whichmay implement embodiments of the invention. The industrial automationsystem 200 is similar to the instrumentation or test and measurementsystem 100 shown in FIG. 2A. Elements which are similar or identical toelements in FIG. 2A have the same reference numerals for convenience.The system 200 may comprise a computer 82 which couples to one or moredevices or instruments. The computer 82 may comprise a CPU, which may beconfigured with various embodiments of the present invention, a displayscreen, memory, and one or more input devices such as a mouse orkeyboard as shown. The computer 82 may operate with the one or moredevices to perform an automation function with respect to a process ordevice 150, such as MMI (Man Machine Interface), SCADA (SupervisoryControl and Data Acquisition), portable or distributed data acquisition,process control, advanced analysis, or other control, among others,e.g., via execution of software 104.

The one or more devices may include a data acquisition board 114inserted into or otherwise coupled with chassis 124 with associatedsignal conditioning circuitry 126, a PXI instrument 118, a video device132 and associated image acquisition card 134, a motion control device136 and associated motion control interface card 138, a fieldbus device270 and associated fieldbus interface card 172, a PLC (ProgrammableLogic Controller) 176, a serial instrument 282 and associated serialinterface card 184, or a distributed data acquisition system, such asthe Fieldpoint system available from National Instruments, among othertypes of devices.

FIG. 3 illustrates an exemplary system 300 for scheduling a timecritical task. The exemplary system may include a processing unit 310.Note, while the exemplary system may include a processing unit, it isenvisioned that any of a variety of processing elements, or functionalunits, may be used in the system for scheduling a time critical task.The processing unit 310 may be coupled to a memory, such as systemmemory 320. Further, the system memory 320 may be coupled to a devicesuch as hardware assist scheduler 340. The hardware assist scheduler 340may be implemented on any type of processing element, including as asecond processing unit, either as an integral part of the computingsystem or as a peripheral device communicatively coupled to the computersystem via an interconnect. The hardware assist scheduler 340 may beconfigured to receive timing information for executing the time criticaltask. The time critical task may execute program instructions via athread on a core a processing unit, such as processing unit 310. Notethat a thread refers to the smallest sequence of programminginstructions that an operating system scheduler can manageindependently. The hardware assist scheduler 340 may schedule the timecritical task by determining, based on the received timing information,a wakeup time, a thread cycle time, an overall thread deadline, and atime critical task deadline. In certain embodiments, the overall threaddeadline may be the difference between the thread cycle time and asystem dependent suspend latency, e.g., the system specific timerequired to suspend the thread. In some embodiments, the time criticaltask deadline may be a function of the overall thread deadline and aspecified thread bandwidth. In an exemplary embodiment, the threadbandwidth may indicate a fraction of the thread cycle time reserved forthe time critical task. In other embodiments, the thread bandwidth mayindicate a percentage of the thread cycle time reserved for the timecritical task.

Further, the hardware assist scheduler 340 may program a timer, referredto herein as a lateness timer for convenience, based on the wakeup timeand the time critical task deadline and wait for the wakeup time toobtain, e.g., wait for the condition of the wakeup time to be achieved.In other words, the hardware assist scheduler 340 will monitor the timeand wait for a specified time, e.g., the wakeup time, to arrive.Additionally, the hardware assist scheduler 340 may be configured tonotify the processing unit 310 of the schedule. To notify the processingunit 310, the hardware assist scheduler 340 may be configured toindicate to the processing unit 310 that the thread is in an idle state,where the idle state of the thread may indicate that the thread is awakeand ready to resume execution and execute the time critical task inaccordance with the schedule.

The processing unit 310 may be configured to resume execution of thethread upon notification that the thread is in the idle state andexecute the time critical task on the thread. The hardware assistscheduler 340 may be further configured to monitor the lateness timerand assert a thread interrupt in response to the lateness time expiring.Additionally, the hardware assist scheduler 340, in order to assert thethread interrupt, may be configured to interrupt execution of the threadon the core of the processing unit 310 causing the execution of the timecritical task to be suspended. In other words, the interrupt is onlyused to stop the execution of a task and never used to start theexecution of a task. In certain embodiments, the hardware assistscheduler 340, in order to suspend the execution of the time criticaltask, may be further configured to suspend the execution of the threadand return the thread to the idle state, ensuring a timely wakeup of thethread for execution of another cycle of the time critical task.

In some embodiments, the thread may include a multiplexed thread, e.g.,a thread programmed to execute multiple tasks sequentially. Themultiplexed thread may be executed on the core of a processing unit,such as processing unit 310. In an exemplary embodiment, the thread mayinclude a plurality of time critical tasks, including the time criticaltask discussed previously. Further, each of the plurality of timecritical tasks may have respective timing information and may each beexecuted in accordance with the respective timing information, e.g.,each respective time critical task may have respective timinginformation such that the hardware assist scheduler 340 may scheduleeach respective time critical task by determining, based on eachrespective received timing information, a respective wakeup time, arespective thread cycle time, a respective overall thread deadline, anda respective time critical task deadline. In other embodiments where thethread comprises a multiplexed thread that may be executed on the coreof the processing system, the multiplexed thread may include one or moretime critical tasks and a best effort task. A best effort task refers toa task in a real time computer system used to execute programinstructions that do not have a timeliness requirement, e.g., does notneed to meet a deadline. Examples include, but are not limited to,non-time critical programs such as logging status, responding to userinterface commands, monitoring health of the system, and so forth.

In some embodiments involving best effort tasks, the hardware assistscheduler 340 may be further configured to wake up the best effort taskin response to the lateness timer expiring and notify the processingunit 310 that the best effort task is awake and ready to be executed.Additionally, the hardware assist scheduler 340 may be furtherconfigured to reset the lateness time, suspend the thread, and returnthe thread to the idle state, ensuring a timely wakeup of the thread forexecution of another cycle of the time critical task.

In certain embodiments, the hardware assist scheduler 340 may be furtherconfigured to receive timing information for executing the best efforttask. The hardware assist scheduler 340 may then be configured toschedule the best effort task by determining, based on the receivedtiming information, a start time for the best effort task where thestart time for the best effort task is later than the time critical taskdeadline. Specification of a precise start time for the best efforttasks may allow the hardware assist scheduler 340 to offset access toshared resources, such as the processing unit's cache, the system memory320, and so forth.

In other embodiments involving best effort tasks, the thread may be oneof a plurality of threads executing on the core of the processing unit310. In such embodiments, a thread of the plurality of threads may bededicated to execution of best effort tasks. In such instances, thehardware assist scheduler 340 may be used to trigger the execution ofthe best effort tasks at a specified start time such that the executionof the best effort tasks does not conflict with time critical tasksbeing executed on another one or more of the plurality of threadsexecuting on the core of the processing unit 310.

FIG. 4 illustrates an exemplary system 400 for scheduling a timecritical task according to an embodiment of the present invention. Insystem 400, processing unit 310, as described in detail above, may becoupled to system memory 320 via memory controller 455. Additionally,hardware assist scheduler 440 may be coupled to system memory 320 viamemory controller 455.

Hardware assist scheduler 440 may incorporate all of the functionalitydiscussed above with regards to hardware assist scheduler 340. Inaddition, in some embodiments, hardware assist scheduler 440, in orderto notify the processing unit 310 of the schedule, may be furtherconfigured to perform a memory write to system memory 320 via memorycontroller 455. Further, the processing unit 310 may be furtherconfigured to monitor the system memory for the memory write. This socalled “memory monitoring methodology” is fully described in pendingU.S. patent application Ser. No. 13/594,088, entitled “Hardware AssistedReal-time Scheduler Using Memory Monitoring” filed on Aug. 24, 2012, andfully incorporated herein. In brief, the hardware assist scheduler 440,in certain embodiments, may be further configured to generate or modifya schedule of one or more tasks, causing a new schedule to be createdand write to a specified location in the memory, indicating that the newschedule has been created. Further, the processing unit 310 may befurther configured to monitor for a write to the specified location inthe memory and execute the one or more tasks in accordance with the newschedule in response to detecting the write to the specified location inthe memory.

FIG. 5 illustrates an exemplary system 500 for scheduling a timecritical task according to an embodiment of the present invention. Insystem 500, the hardware assist scheduler 540, which may incorporate allof the features fully described above in reference to hardware scheduler340, including being coupled to system memory 320, may be furtherconfigured to provide a global clock signal for the system 500 via clock545. Additionally, processing unit 510, in addition to incorporating allof the features fully described above in reference to processing unit310, may be further configured to perform a timed thread execution. Theprocessing unit 510, in order to perform the timed thread execution maybe configured to synchronize, via a time monitor circuit, a clock 515 ofthe processing unit with the global clock 545, monitor, via the timemonitor circuit, the clock of the processing unit with respect to thewakeup time, and determine that the wakeup time obtains. Further, theprocessing unit 510, may, in response to the wakeup time obtaining,resume the execution of the thread. The “time monitor circuit” and usethereof, is fully described in pending U.S. Provisional PatentApplication No. 61/714,006, entitled “Time Monitoring in a ProcessingElement and Use” filed on Oct. 15, 2012, and fully incorporated herein.In brief, in some embodiments, processing unit 510 may include clock515, a thread suspend/resume logic block 535, and a time monitor circuit525. The thread suspend/resume logic block 535, in some embodiments, maybe configured to suspend execution of a thread in response to a receivedsuspend thread instruction. Further, the time monitor circuit 525 may beconfigured to receive an indication of a wakeup time, monitor time viaclock 515 until the wakeup time obtains, and invoke the threadsuspend/resume logic block 535 to resume execution of the thread inresponse to the wakeup time obtaining. In other words, the time monitorcircuit 525 may utilize clock 515 of the processing element 510 todetermine when the wakeup time has arrived and resume execution of thethread.

In other embodiments, clock 545 of hardware assist scheduler 540 may beconfigured to synchronize with a global clock. In such embodiments,clock 515 of processing unit 510 would be configured to synchronize withthe global clock such that clock 515 would be synchronized in phase andfrequency with clock 545 via synchronization to a global clock.

FIG. 6 illustrates an exemplary multicore processing unit 610 forscheduling and executing time critical tasks. Processing unit 610 mayinclude a plurality of cores, e.g., cores 630 a, 630 b, and so forththrough core 630 n, and collectively referred to as cores 630. Cores 630may be coupled to local cache 645. In some embodiments, a hardwareassist scheduler, as fully described above in reference to hardwareassist schedulers 340, 440, and 540 may be included on a core of theplurality of cores 630, e.g., core 630 a. Further, the time criticaltasks may execute on at least one other core of the plurality of cores630 of the processing unit 610. For example, the hardware assistscheduler may be included on core 630 a while the time critical tasksmay be executed on core 630 b, or cores 630 b through 630 n. In someembodiments, where the hardware assist scheduler is included on a coreof the processing unit 610, the memory monitor methodology as describedabove with reference to FIG. 4 may be fully incorporated into processingunit 610. Further, in other embodiments, where the hardware assistscheduler is included on a core of the processing unit 610, the timemonitor circuit as described above with reference to FIG. 5 may be fullyincorporated into processing unit 610.

FIG. 7 illustrates an exemplary system 700 for scheduling a timecritical task according to an embodiment of the present invention. Insystem 700, processing unit 710, which may configured to as describedabove with regards to processing units 310 and 510, may becommunicatively coupled to a system memory 320 via a memory controller755. Further, memory controller 755 may be communicatively coupled to amemory-mapped interconnect 760. In some embodiments, the memory-mappedinterconnect 760 may include a high speed interconnect. Examples of highspeed interconnects include PCI Express (PCIe) switches, RapidIOswitches, 100G Ethernet (100 GbE), active optical cables for Ethernet,QUICKPATH interconnect (QPI) or HYPERTRANSPORT® interconnect andThunderbolt. In other embodiments, memory-mapped interconnect 760 mayinclude a photonics interconnect, e.g., a high speed fiber opticinterconnect.

Hardware assist scheduler 740 may be communicatively coupled to thesystem memory via the memory-mapped interconnect 760 and the memorycontroller 755. In some embodiments, the hardware assist scheduler maybe configured as described above in reference to hardware assistscheduler 340 of FIG. 3. In other embodiments, hardware assist scheduler740 may be configured as described above in reference to hardware assistscheduler 440 of FIG. 4. Additionally, in yet other embodiments,hardware assist scheduler 740 may be configured as described above inreference to hardware assist scheduler 540.

FIG. 8—Flowchart

FIG. 8 illustrates a method for monitoring a time critical task. Note,that the method described below may be fully implemented using any ofthe above described systems and embodiments. While the method is fullydescribed, it must be noted that not all steps may be necessary or maybe performed in an order other than described below.

In 802, timing information for executing the time critical task may bereceived. The time critical task may execute program instructions via athread on a core of a processing unit, such as processing units 310,510, 610, and 710 described in detail above in reference to FIGS. 3through 7.

In 804, the time critical task may be scheduled. Scheduling the timecritical task may include determining, based on the received timinginformation, a wakeup time, a thread cycle time, an overall threaddeadline, and a time critical task deadline. In some embodiments, theoverall thread deadline may be the difference between the thread cycletime and a system dependent suspend latency.

In 806, a lateness time based on the wakeup time and the time criticaltask deadline may be programmed. In certain embodiments, the timecritical task deadline may be a function of the overall thread deadlineand a specified thread bandwidth, where the specified thread bandwidthmay indicate a fraction of the thread cycle time reserved for the timecritical task. In other embodiments, the time critical task deadline maybe a function of the overall thread deadline and a specified threadbandwidth, where the specified thread bandwidth may indicate apercentage of the thread cycle time reserved for the time critical task.

In 808, the obtainment of the wakeup time is waited for. In other words,the arrival of the wakeup time is awaited.

In 810, the processing unit is notified of the schedule, where thenotifying indicates to the processing unit that the thread is in an idlestate. The idle state indicates that the thread is awake and ready toresume execution and execute the time critical task in accordance withthe schedule. In some embodiments, the notifying may include a memorywrite to a memory of the processing unit and the processing unitmonitoring the memory for a write to indicate the notification. In suchembodiments, the methods described above in reference to U.S. patentapplication Ser. No. 13/594,088, entitled “Hardware Assisted Real-timeScheduler Using Memory Monitoring” filed on Aug. 24, 2012 may beincorporated. In other embodiments, the notifying may include a timedthread execution where the timed thread execution includessynchronizing, via a time monitor circuit, as fully described in U.S.Provisional Patent Application No. 61/714,006, entitled “Time Monitoringin a Processing Element and Use” filed on Oct. 15, 2012, with a globalclock and monitoring, by the processing unit via the time monitorcircuit, the clock of the processing unit with respect to the wakeuptime. Additionally, in such embodiments, the processing unit maydetermine that the wakeup time obtains and the thread execution may beresumed in response to determining that the wakeup time obtained, e.g.,the time to wakeup the thread has arrived.

In 812, the time critical task may be executed on the core of theprocessing unit in accordance with the schedule determined in 804.

In 814, the lateness timer may be monitored.

In 816, in response to the lateness timer expiring, a thread interruptmay be asserted, where asserting the thread interrupt interrupts theexecution of the thread and suspends execution of the time criticaltask. In some embodiments, suspending execution of the time criticaltask may include suspending execution of the thread and returning thethread to the idle state to ensure a timely wakeup of the thread forexecution of another cycle of the time critical task.

In certain embodiments of the above described method, the thread mayinclude a multiplexed thread where the multiplexed thread is executed onthe core of the processing unit. In some embodiments, the multiplexedthread may include a plurality of time critical tasks, including thetime critical task, where each of the plurality of time critical taskshas respective timing information. Thus, each of the plurality of timecritical tasks is executed in accordance with the respective timinginformation. In other embodiments, the multiplexed thread may includeone or more time critical tasks and a best effort task. In suchembodiments, the method may further include waking up the best efforttask in response to the lateness timer expiring and executing the besteffort task until the overall thread deadline obtains. In other words,the best effort tasks, e.g., the non-time critical tasks, may beexecuted until the overall thread deadline arrives. Then, the latenesstimer may be reset, execution of the thread may be suspended, and thethread may be returned to the idle state. Returning the thread to theidle state may ensure a timely wakeup of the thread for execution ofanother cycle of the time critical task.

Further, in certain embodiments involving best effort tasks, timinginformation for executing the best effort task may be received. Thus,the best effort task may be scheduled. The scheduling of the best efforttask may include determining, based on the received timing information,a start time for the best effort task. The start time for the besteffort task may be later than the time critical task deadline.

Although the embodiments above have been described in considerabledetail, numerous variations and modifications will become apparent tothose skilled in the art once the above disclosure is fully appreciated.It is intended that the following claims be interpreted to embrace allsuch variations and modifications.

We claim:
 1. A method for monitoring a time critical task, the methodcomprising: determining a schedule for the time critical task based onreceived timing information, wherein the time critical task executesprogram instructions via a thread on a core of a processing unit,wherein the time critical task is a function of the overall threaddeadline and a specified thread bandwidth, and wherein said determiningcomprises determining a wakeup time, a thread cycle time, and an overallthread deadline; executing, on the core of the processing unit, the timecritical task in accordance with the schedule; and suspending executionof the time critical task via assertion of a thread interrupt inresponse to a lateness timer expiring, wherein the lateness timer isbased on the wakeup time and the time critical task deadline.
 2. Themethod of claim 1, further comprising: notifying the processing unitthat the thread is in an idle state, wherein the idle state indicatesthat the thread is awake and ready to resume execution and execute thetime critical task in accordance with the schedule.
 3. The method ofclaim 2, wherein said notifying comprises a memory write to a memory ofthe processing unit; wherein the method further comprises monitoring thememory, by the processing unit, for the memory write to indicate saidnotifying.
 4. The method of claim 2, wherein said notifying comprises atimed thread execution; wherein the timed thread execution comprises:synchronizing, via a time monitor circuit, a clock of the processingunit with a global clock; monitoring, by the processing unit via thetime monitor circuit, the clock of the processing unit with respect tothe wakeup time; and determining that the wakeup time obtains; andwherein the method further comprises: resuming the execution of thethread in response to said determining that the wakeup time obtains. 5.The method of claim 1, wherein the overall thread deadline is based onthe difference between the thread cycle time and a system dependentsuspend latency.
 6. The method of claim 1, wherein said suspendingexecution of the time critical task comprises: suspending execution ofthe thread; and returning the thread to an idle state, thereby ensuringa timely wakeup of the thread for execution of another cycle of the timecritical task.
 7. The method of claim 1, wherein the time critical taskdeadline is a function of the overall thread deadline and a specifiedthread bandwidth, and wherein the specified thread bandwidth indicates afraction of the thread cycle time reserved for the time critical task.8. The method of claim 1, wherein the thread comprises a multiplexedthread, wherein the multiplexed thread is executed on the core of theprocessing unit, and wherein the multiplexed thread comprises: one ormore time critical tasks; and a best effort task.
 9. A system forscheduling a time critical task, the system comprising: a processingunit; a device configured as a hardware assist scheduler; and a memorycoupled to the processing unit and the hardware assist scheduler;wherein the hardware assist scheduler is configured to: determine aschedule for the time critical task based on received timinginformation, wherein the time critical task executes programinstructions via a thread on a core of the processing unit, wherein thetime critical task is a function of the overall thread deadline and aspecified thread bandwidth, and wherein said determining comprisesdetermining a wakeup time, a thread cycle time, and an overall threaddeadline; and wherein the processing unit is configured to execute thetime critical task in accordance with the schedule; and wherein thehardware assist scheduler is further configured to suspend execution ofthe time critical task via assertion of a thread interrupt in responseto a lateness timer expiring, wherein the lateness timer is based on thewakeup time and the time critical task deadline.
 10. The system of claim9, wherein the hardware assist scheduler is further configured to notifythe processing unit that the thread is in an idle state, and wherein theidle state indicates that the thread is awake and ready to resumeexecution and execute the time critical task in accordance with theschedule.
 11. The system of claim 10, wherein said notifying comprises amemory write to a memory of the processing unit; and wherein theprocessing unit is further configured to monitor the memory for thememory write to indicate said notifying.
 12. The system of claim 9,wherein the hardware assist scheduler is further configured to provide aglobal clock signal; and wherein the processing unit is furtherconfigured to: perform a timed thread execution, wherein to perform thetimed thread execution, the processing unit is configured to:synchronize, via a time monitor circuit, a clock of the processing unitwith the global clock; monitor, via the time monitor circuit, the clockof the processing unit with respect to the wakeup time; and determinethat the wakeup time obtains; and resume the execution of the thread inresponse to the wakeup time obtaining.
 13. The system of claim 9,wherein the overall thread deadline is based on the difference betweenthe thread cycle time and a system dependent suspend latency.
 14. Thesystem of claim 9, wherein, to suspend execution of the time criticaltask, the hardware assist scheduler is further configured to: suspendexecution of the thread; and return the thread to an idle state, therebyensuring a timely wakeup of the thread for execution of another cycle ofthe time critical task.
 15. The system of claim 9, wherein the timecritical task deadline is a function of the overall thread deadline anda specified thread bandwidth, and wherein the specified thread bandwidthindicates a fraction of the thread cycle time reserved for the timecritical task.
 16. The system of claim 9, wherein the thread comprises amultiplexed thread, wherein the multiplexed thread is executed on thecore of the processing unit, wherein the multiplexed thread comprises:one or more time critical tasks; and a best effort task.
 17. Anon-transitory computer readable memory medium storing programinstructions executable by at least one processor to: determine aschedule for a time critical task based on received timing information,wherein the time critical task executes program instructions via athread on a core of a processing unit, wherein the time critical task isa function of the overall thread deadline and a specified threadbandwidth, and wherein said determining comprises determining a wakeuptime, a thread cycle time, and an overall thread deadline; notify theprocessing unit that the thread is in an idle state, wherein the idlestate indicates that the thread is awake and ready to resume executionand execute the time critical task in accordance with the schedule; andsuspend execution of the time critical task via assertion of a threadinterrupt in response to a lateness timer expiring, wherein the latenesstimer is based on the wakeup time and the time critical task deadline.18. The non-transitory computer readable memory medium of claim 17,wherein to notify the processing unit, the program instructions arefurther executable to perform a memory write to a memory of theprocessing unit.
 19. The non-transitory computer readable memory mediumof claim 17, wherein the overall thread deadline is based on thedifference between the thread cycle time and a system dependent suspendlatency.
 20. The non-transitory computer readable memory medium of claim17, wherein the time critical task deadline is a function of the overallthread deadline and a specified thread bandwidth, and wherein thespecified thread bandwidth indicates a fraction of the thread cycle timereserved for the time critical task.